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SUMMARY 

In  anti-air  warfare  (AAW),  rapidly  evolving  situations  require  very  fast  decisions  which  are  extremely 
difficult  to  make,  especially  in  so-called  “mixed  environments  ’’  where  neutral,  friendly,  and  hostile  objects 
are  present.  Thus  the  importance  of  supporting  AA  W  decision  makers  increases  significantly.  The  basis  of 
any  support  system  must  be  an  ergonomically  designed  user  interface  covering  all  stages  of  human 
information  processing.  Additionally,  improving  the  cognitive  processes  of  operators  by  computer-based 
decision  proposals  is  important.  To  achieve  this,  techniques  beyond  rule-based  “artificial  intelligence” 
(AI)  are  considered,  such  as  Bayesian  belief  networks,  case-based  reasoning,  and  fuzzy  theory.  To  be  able 
to  operate  advanced  AI  methods  fruitfully,  it  is  necessary  to  acquire,  describe,  formalize,  and  represent 
knowledge  and  problem  solving  strategies  of  domain  experts.  The  first  step  is  the  identification  of  decision 
making  variables  by  means  of  an  empirical  knowledge  acquisition.  Military  experts  will  be  observed 
when  making  decisions  and  asked  to  substantiate  them  while  facing  simulated  scenarios.  The  knowledge 
acquired  this  way  is  needed  for  case  bases,  which  intelligent  agents  can  access,  as  well  as  for  an  analysis 
of  operators  ’  information  demands. 


1.0  INTRODUCTION 

As  the  scope  of  military  duties  shifts  more  and  more  from  pure  national  defense  towards  participating  in 
multinational  peace-keeping  missions,  future  command  and  control  (C^)  systems  have  to  meet  the  specific 
requirements  of  such  “out  of  area”  missions,  which  are  characterized  by  eminently  high  complexity  and 
uncertainty  in  the  tasks  of  situation  recognition  and  assessment,  as  well  as  in  the  choice  of  adequate 
measures.  The  presence  of  neutral,  friendly,  and  hostile  objects  necessitates  non-ambiguous  identification. 
Thereby,  the  identification  of  unknown  objects,  the  detection  of  the  abilities  and  intentions  of  possibly 
threatening  objects,  the  interpretation  of  rules  of  engagement,  as  well  as  the  ultimate  decision  on  whether 
or  not  to  engage  a  particular  object  lead  to  exceedingly  difficult  decision  making  situations. 

Besides  these  new  types  of  missions  there  is  also  a  lot  of  technological  advancement.  Range,  speed, 
and  accuracy  of  weapons  increase,  and  so  do  sensors  and  communication  facilities.  The  amount  of 
available  data  about  the  operational  area  has  noticeably  expanded,  e.g.,  by  satellites  and  AWACS. 
Figure  1  shows  an  overview  of  some  factors  relevant  to  tactical  decision  making  onboard  naval  vessels. 
To  emphasize  the  increasing  importance  of  communications,  computers,  intelligence,  surveillance, 
and  reconnaissance  in  the  term  C"^ISR  has  arisen. 
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Figure  1:  Overview  of  the  Probiem  Space  [Boiier  2001], 


Consequently  human  deeision  makers  are  confronted  with  an  increasing  amount  of  information  but  have 
less  time  remaining  to  ponder.  Especially  the  rapidly  changing  situations  in  the  AAW  domain  demand 
decisions  be  made  and,  if  considered  necessary,  actions  be  taken  extremely  fast.  This  means  high 
workload  and  stress  for  the  human  decision  makers  and  will  possibly  result  in  wrong  decisions  with 
serious  consequences.  Thus  the  importance  of  supporting  decision  makers  to  reduce  workload  and 
therewith  improve  planning,  decision  making,  and  operation  safety  increases  significantly. 


2.0  GUIDELINES  FOR  BUILDING  AN  ADAPTIVE  SUPPORT  SYSTEM 

The  basis  of  any  support  system  must  be  an  ergonomically  optimized  user  interface  that  features  task-  and 
user-adapted  handling  and  supports  all  stages  of  human  information  processing.  Wickens’  model  of 
multiple  resources  [Wickens  1991]  implies  that  humans  possess  different  resources  for  the  various 
successive  stages  of  information  processing: 

•  Perception, 

•  Cognition,  and 

•  Sensorimotor  reaction. 

Therefore,  a  goal  of  the  design  of  a  user  interface  must  be  to  reach  an  equable  exploitation  of  the  resources 
being  available  for  these  different  stages.  In  complex  man-machine  systems  the  necessity  to  optimally 
exploit  the  operator’s  resources  creates  demand  for  software-based  support  of  all  mentioned  stages  of 
information  processing.  This  can  be  attained  by  a  knowledge-based  assistant  system  inserted  between  the 
visualization  part  of  the  user  interface  and  the  technical  system  as  shown  in  Figure  2. 
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Figure  2:  Starting  Points  for  Adaptive  User  Support. 

First,  the  concept  for  a  knowledge-based  support  system  must  provide  a  component  to  support  system 
usage.  The  modules  integrated  herein  shall  support  all  stages  of  information  collection,  processing, 
and  transformation  in  a  way  so  that  operators  can  interact  with  the  system  intuitively  and  reach  an 
adequate  situational  awareness. 

Second,  besides  an  improved  design  of  the  man-machine  dialog,  supporting  the  cognitive  processes  of 
operators  by  computer-based  means  becomes  ever  more  important.  Flence  the  authors’  support  concept  is 
primarily  intended  to  provide  agents  that  derive  situation  assessment  proposals  based  on  the  available 
information.  Thereby  the  situation  assessment  module  is  to  influence  all  other  modules  to  achieve  an 
adequate  support  for  each  task  in  any  situation.  The  assistance  system  shall  not  replace  the  operator  but 
rather  serve  as  a  cognitive  amplifier. 


Table  1:  Scale  of  Degrees  of  Automation  [Sheridan  1992] 


Degree  of  Automation 

System  Features 

1 

The  computer  offers  no  assistance,  human  must  do  it  all. 

2 

The  computer  offers  a  complete  set  of  action  alternatives,  and 

3 

narrows  the  selection  down  to  a  few,  or 

4 

suggests  one,  and 

5 

executes  that  suggestion  if  the  human  approves,  or 

6 

allows  the  human  a  restricted  time  to  veto  before  automatic  execution,  or 

7 

executes  automatically,  then  necessarily  informs  the  human,  or 

8 

informs  him  after  execution  only  if  he  asks,  or 

9 

informs  him  after  execution  if  it,  the  computer,  decides  to. 

10 

The  computer  decides  everything  and  acts  autonomously,  ignoring  the  human. 

Sheridan  [Sheridan  1992]  differentiates  ten  degrees  of  automation,  as  shown  in  Table  1.  Level  1  would 
mean  not  to  provide  any  decision  support  at  all.  Levels  8-10  are  not  eligible  for  tactical  decision  making 
because  in  these  approaches  operators  are  completely  dismissed  from  the  loop.  Level  7  can  be  regarded 
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acceptable  only  for  the  initiation  of  so-called  “last-chance”  defenses  against  very  critical  threats,  such  as 
an  inbound  anti-ship  missile.  The  support  provided  to  operators  should  normally  reside  among  levels  2  to 
6  and  adapt  according  to  the  complexity  of  the  task  at  hand,  the  over  all  situation,  and  the  operator  state, 
as  far  as  it  is  possible  to  assess  it. 

3.0  ASPECTS  OF  HUMAN  DECISION  MAKING 

3.1  The  Impact  of  Information  on  Performance 

The  following  primitives  have  been  identified  by  Alberts  et  al.  [Alberts  2001]  as  being  needed  to 
investigate  how  information  affects  performance: 

•  Sensing:  direct  sensing  happens  with  humans’  senses  (e.g.,  seeing).  Visual  recognition  of  an  air 
track  is  an  example  for  direct  sensing  from  the  treated  domain.  However,  most  identification  relies 
on  indirect  sensing,  such  as  radio  detection  and  ranging  (radar). 

•  Data:  data  are  the  raw  form  of  what  has  been  received  by  any  sensing  device. 

•  Information:  the  result  of  merging  individual  pieces  of  data  into  meaningful  context. 
Some  observations  can  be  lost,  due  to  filtering  by  perceptual  lenses. 

•  Knowledge:  the  available  information  inspires  patterns  from  which  conclusions  are  drawn. 
Knowledge  about  the  situation  results  from  these  conclusions.  This  procedure  is  influenced  by 
knowledge  resulting  from  training,  experience,  and  interaction  with  other  humans. 

•  Awareness:  the  result  of  combining  prior  knowledge,  beliefs,  and  current  perceptions  related  to 
the  situation.  It  is  unique  to  any  individual;  nevertheless,  military  training  aims  at  obtaining  a 
common  awareness. 

•  Understanding:  sufficient  awareness  and  knowledge  to  assess  possible  future  patterns  and 
consequences  of  the  situation  constitute  understanding.  It  focuses  on  what  the  situation  can 
become  as  well  as  on  the  impact  of  making  different  decisions. 

•  Decisions:  the  choices  about  what  to  do.  Some  decisions  imply  actions  to  fulfill.  Decisions  should 
be  based  on  understanding;  however,  they  can  of  course  also  be  made  without  understanding, 
which  is  undoubtedly  the  case  to  be  avoided. 

•  Actions:  some  decisions  being  made  trigger  actions  to  be  executed. 

Because  this  paper  concentrates  exclusively  on  single  operator  tasks  in  AAW,  sharing,  collaboration,  and 
synchronization  have  been  skipped. 

3.2  Threat  Evaluation  on  the  Basis  of  Observed  Activities 

The  following  definitions  are  required  to  model  AAW  decision  making  processes: 

•  Activities:  the  behavior  of  the  track  that  is  actually  observable,  for  instance  its  suitability  to  an 
airway,  its  maneuvers,  or  weapon  releases. 

•  Options:  all  activities  the  track  is  able  to  perform  in  the  near  future  referring  to  its  current 
situation. 

•  Intention:  the  long  term  mission  goal  of  the  aircraft’s  pilot.  The  AAW-operator,  taking  into 
consideration  the  activities  observed  up  to  the  present,  can  merely  guess  which  of  the  imaginable 
intentions  is  most  probable  and  which  of  its  options  the  track  is  therefore  most  likely  to  realize. 

•  Threat:  depending  on  the  detected  intention,  in  a  threat  evaluation  the  likelihood  of  an  attack  on 
own  forces  among  other  things  is  analyzed.  The  result  can  lead  to  an  engagement. 
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Figure  3  shows  an  example  of  how  options,  likeliest  intentions,  and  threat  assessment  evolve  over  time 
with  developing  aetivities  of  a  traek. 
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Figure  3:  From  Activities  to  Threat  Evaiuation,  a  Refinement  foiiowing  Doring  et  ai  [Doring  2002], 


3,3  The  Three  Levels  of  Decision  Making  Performance 

As  shown  in  Figure  4,  deeision  making  ean  be  divided  into  the  steps  “situation  assessment”  and 
“solution  generation”  [Ddrfel  1999].  Aeeording  to  Rasmussen  [Rasmussen  1983],  information  proeessing, 
as  indicated  by  the  cognitive  level  on  which  a  decision  is  made,  can  be  assigned  to: 

•  Skill-based  behavior:  predetermined  signals  are  recognized  and  interrelated  directly  with  one  or 
more  actions  to  be  performed. 

•  Rule-based  behavior:  situations  are  interpreted  according  to  rules.  Therefore,  knowledge  is 
extracted  from  the  situation.  Actions  are  triggered  depending  on  the  ascertained  situation. 
The  action  planning  results  from  the  existing  system  knowledge  that  determines  the  manner  of 
potential  actions. 

•  Knowledge-based  behavior:  if  there  are  no  rules  present  for  a  situation,  it  thus  represents  a  totally 
new  experience,  it  is  tried  to  assess  the  situation  by  calling  in  experiences  from  the  past. 
Structuring  on  this  and  having  regard  to  overall  mission  objectives  a  goal  is  defined  and 
goal-directed  action  plans  are  formulated. 
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Figure  4:  The  Three  Levels  of  Performance  [Dorfel  1999]. 
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By  utilizing  computer-based  decision  support  systems  it  can  be  tried  to  emulate  the  corresponding  human 
decision  making  processes  closely  matching. 


4.0  VISUALIZATION  ASPECTS 

As  pointed  out  in  the  introduction  as  well  as  in  Figure  1,  AAW  operators  onboard  naval  vessels  have  to 
make  decisions  based  on  a  multitude  of  data  and  information.  In  current  systems  these  are 
predominantly  presented  in  alphanumeric  form.  According  to  Card  [Card  1999]  a  user-centered 
presentation  of  complex  data  should  feature  the  use  of  computer-supported,  interactive, 
visual  representations  to  amplify  cognition  by: 

•  increasing  the  memory  and  processing  resources  available  to  operators, 

•  reducing  the  search  for  information, 

•  using  visual  representations  to  enhance  the  detection  of  patterns, 

•  enabling  perceptual  inference  operations, 

•  using  perceptual  attention  mechanisms  for  monitoring,  and 

•  encoding  information  in  a  manipulable  medium. 

Some  inferences  can  be  drawn  easily  due  to  graphical  visualization,  whereas  without  it  would  be  more 
difficult.  By  appropriately  mapping  information  into  visual  form  cognitive  effort  can  be  enhanced. 
The  over  all  idea  is  to  support  thinking  by  visual  perception. 

Figure  5  shows  two  states  of  a  safety  parameter  monitoring  display  [Wickens  1997]  designed  to  support 
nuclear  power  plant  operators.  A  polygon  connects  the  values,  indicated  by  the  distance  from  the  center, 
of  eight  parameters.  As  long  as  everything  is  all  right  a  harmonic  uniform  polygon,  as  the  one  on  the  left, 
originates  from  the  data.  The  right  one  indicates  that  there  are  parameters  that  deviate  from  normal. 
Similar  visualization  techniques  can  be  developed  to  support  the  information  collection,  processing, 
and  transformation  phases  of  AAW  operators  [Pfendler  1999].  Finding  an  adequate  way  to  provide  them 
with  all  relevant  information  is  an  essential  with  the  development  of  future  systems. 


CORE  EXIT  CORE  EXIT 

618/618°F  579/422°F 


Figure  5:  Integrated  Polar  Display  for  Nuclear  Power  [Wickens  1997], 
the  right  one  indicates  Parameter  Deviations. 

5.0  FORMAL  DECISION  SUPPORT  TECHNIQUES 

The  main  goal  of  the  aspired  support  system  is  to  draw  conclusions  from  the  data  on  hand  regarding  all 
accumulated  knowledge.  To  achieve  a  sufficient  quality  of  reasoning  from  an  automatic  system. 
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mathematically  consistent  inference  techniques  are  needed.  Many  currently  realized  computerized  support 
systems  try  to  fully  describe  all  anticipated  situations  as  rules.  Such  rules  act  upon  Boolean  logic  and 
deduce  or  exclude  solutions  assuredly.  There  cannot  be  anything  in-between  at  all. 

In  contrast,  military  situations  can  often  be  characterized  by  one  or  more  of  the  following  aspects: 

•  They  cannot  be  described  on  the  basis  of  univocal  rules.  These  kind  of  situations  can  only  be 
described  and  solved  by  calling  in  previously  acquired  know-how. 

•  They  are  not  conform  to  any  expectations  that  have  been  considered  within  the  system 
development  process  and  are  not  accounted  for  in  the  system  concept  therefore. 

•  Because  of  physical  and  especially  tactical  restrictions  with  the  use  of  sensors  a  complete  and  up 
to  date  set  of  data  does  not  necessarily  exist  for  all  objects  in  the  operational  environment. 

•  They  allow  several  inconsistent  interpretations.  In  the  AAW  domain  this  can  arise,  for  instance, 
from  the  fact  that  in  terms  of  speed,  course,  and  altitude  an  enemy  warplane  can  behave  similarly 
to  a  neutral  cargo  plane. 


According  to  the  introduced  levels  of  decision  making  performance,  a  purely  rule-based  support  system  is 
not  capable  of  supporting  the  human  decision  maker  in  a  situation  in  which  knowledge-based  acting  is 
essential.  Therefore,  approaches  beyond  rule-based  knowledge  representation  have  to  be  taken  into 
account. 


5.1  Description  of  Techniques 

5.1.1  Boolean  Inference 
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Figure  6:  Example  Decision  Tree  and  -Table. 


The  previous  elucidation  has  made  clear  that  in  military  environments  there  are  situations  that  purely 
rule-based  systems  cannot  appropriately  deal  with.  Nevertheless,  constellations  in  which  no  other  support 
technique  can  outperform  Boolean  inference  do  exist  as  well.  For  instance,  no  airliner  will  ever  have  a  turn 
rate  of  more  than  about  1  degree  per  second.  Not  only  that  it  could  not  be  expected  of  the  passengers. 
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in  fact  big  planes,  which  are  usually  deployed  for  passenger  transportation,  are  not  even  capable  of 
performing  such  a  rate.  Therefore,  if  a  particularly  high  turn  rate  is  observed,  the  hypothesis  that  the  track 
could  be  non-military  can  be  excluded. 

There  are  two  variants  for  the  implementation  of  such  rules:  decision  trees  and  -tables  (Figure  6).  Decision 
trees  consist  of  inner  knots  that  contain  questions,  edges  meaning  answer  alternatives,  and  leaves  that  can 
be  solutions  or  invocations  of  other  decision  trees.  A  tree  is  processed  by  alternately  asking  a  question  and 
marking  out  a  solution  or  the  next  question  by  applying  the  associated  rules.  Trees  are  capable  of 
representing  knowledge  in  a  quite  compressed  manner.  Complex  decision  trees  are  difficult  to  alter  due  to 
the  inherent  dependencies.  In  contrast,  decision  tables  represent  rules  independently  and  therewith  allow 
very  modular  structuring.  Because  of  this  modularity,  it  might  be  necessary  to  implement  complex 
preconditions  as  well  as  logical  nesting. 

5.1.2  Fuzzy  Sets 

In  contrast  to  decision  tables  with  sets  of  Boolean  rules,  fuzzy  logic  [Seising  1 999]  can  be  applied  to  tasks 
in  which  the  degree  of  conformance  can  neither  be  set  assuredly  “true”  nor  “false”  but  merely  be  mapped 
to  fuzzy  sets.  Such  “fuzzy  classifiers”  are  based  on  normative  membership  functions  related  to  the  object 
attributes  to  be  classified  (Figure  7). 


Figure  7:  An  Exemplary  Fuzzy  Classifier. 


Throughout  the  fuzzification,  the  probability  for  an  object  attribute  of  belonging  to  a  certain  valuation 
class,  e.g.,  helicopter  /  airliner  /  warplane  or  suitable  to  airway  /  not  suitable  to  airway  is  stated. 
All  discrete  classifiers  can  be  stochastically  combined,  for  instance,  using  Bayesian  belief  networks,  which 
are  described  beneath.  Therewith  likelihood  ratios  about  the  type  of  an  object  and  the  threat  emanated 
from  it  can  be  deduced.  The  membership  functions  of  classic  fuzzy  classifiers  are  static,  whereas  neuro- 
fuzzy  techniques  are  capable  of  readjusting  classifiers  by  modulating  the  applied  membership  functions. 

5.1.3  Bayesian  Belief  Networks 

The  basic  idea  of  Bayesian  belief  networks  [Mitchell  1997]  is  very  similar  to  heuristic  classification  with 
the  difference  that  relationships  between  attributes  and  solutions  are  not  estimated  by  experts  but  rather 
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extracted  from  case  databases.  This  form  of  statistical  classification  is  suitable  for  problems  for  which  a 
substantial,  representative  collection  of  successfully  solved  cases  is  available.  Its  most  important 
advantage  is  that  it  provides  the  objectifiability  of  knowledge.  The  following  requirements  have  to  be 
fulfilled: 

•  Alternating  Markovian  independence  of  attributes 

•  Completeness  of  the  set  of  solutions 

•  Alternating  exclusion  of  solutions 

•  Representativeness  of  the  case  collection 

•  Availability  of  a  sufficient  set  of  cases  for  any  alternative  solution 

The  theorem  of  Bayes  requires  the  a  priori  probability  of  any  solution  and  the  conditional  probability  of 
any  attribute.  Both  kinds  of  probabilities  can  be  calculated  from  a  case  collection.  Computerized 
procedures  to  model  networks  based  on  case  collections  are  available  as  well. 


Figure  8:  An  Exemplary  Trained  Bayesian  Belief  Network. 


Figure  8  shows  an  exemplary  Bayesian  belief  network  for  the  AAW  domain.  The  knots  show  the 
conditional  probabilities  of  occurrence  for  any  parameter  value  of  an  attribute,  given  all  of  its 
predecessors.  The  omitted  knots  at  the  top,  indicated  by  the  arrows,  stand  for  a  yet  unknown  number  of 
preconditions  that  influence  the  likelihood  distributions  of  the  other  variables,  such  as  the  political 
situation,  the  surrounding  landscape,  or  the  weather. 


5.1.4  Case-Based  Reasoning 

Instance-based  methods,  such  as  case-based  reasoning  [Puppe  1 996]  are  so-called  lazy  learning  methods, 
because  they  do  not  generalize  beyond  their  accumulated  data  unless  a  new  classification  problem 
emerges.  Once  a  new  case  occurs,  similar  instances  are  analyzed.  If  the  nearest  found  case  is  similar 
enough,  subject  to  the  condition  that  there  is  no  comparable  case  with  an  entirely  different  solution  to  the 
new  one,  its  solution  is  taken  over.  Thereby  the  respectively  corresponding  characteristics  of  cases  are 
compared,  and  a  similarity  index  is  calculated  out  of  the  individual  comparisons.  Case-based  classification 
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is  eligible  for  classifieation  problems  for  whieh  enfolded  eollections  of  real  or  made  up  eases  with  eorreet 
solutions  and  detailed  logging  of  all  attributes  exist. 

In  prineiple  the  ease  on  hand  must  be  compared  to  every  known  case  in  the  database  to  find  out 
which  one  is  the  most  similar.  A  means  to  reduce  this  effort  is  to  assemble  very  similar  cases  to  clusters 
that  are  represented  by  joint  agents.  Furthermore,  pre-selection  strategies  could  be  advantageous, 
e.g.,  pre-selection  on  the  basis  of  particularly  important  attributes  and  hill-climbing  via  case  neighborhood, 
whereat  closest  neighbors  are  checked  until  the  utterly  most  similar  one  has  been  found. 

5.1.5  Artificial  Neural  Networks 

Artificial  neural  networks  (ANN)  [Schoneburg  1990]  simulate  the  way  biological  nervous  systems  are 
believed  to  work.  Namely  the  human  brain  structurally  consists  of  neurons.  ANN  are  based  on  simulated 
neurons  which  are  joined  to  networks  in  many  different  ways.  These  networks  are  able  to  create 
relationships  among  data  and  generalize  from  stored  examples  to  new  cases.  Thus  they  have  the  capability 
to  acquire  knowledge  by  learning  from  prior  experiences,  similar  to  adaptive  biological  learning. 
Furthermore  ANN  can  deal  with  imprecise,  probabilistic,  and  fuzzy  data.  For  the  time  being,  typical  fields 
of  application  include  the  recognition  of  speech,  characters,  and  signals. 

In  contrast  to  conventional  algorithmic  approaches,  in  which  inference  is  performed  by  pre-determined 
steps  and  data  are  stored  at  specific  locations,  neural  networks  use  highly  distributed  representations  and 
transformations  that  operate  collaterally.  The  inherent  high  grade  of  parallel  processing  has  led  to  the 
construction  of  special  parallel  processing  hardware  to  facilitate  the  implementation  of  particularly 
efficient  ANN.  Unfortunately  neural  networks  cannot  comprehensibly  communicate  the  path  leading  to  a 
particular  solution  nor  even  give  any  explanation  at  all. 

5.1.6  Genetic  Algorithms 

The  idea  of  genetic  algorithms  (GA)  [Mitchell  1997]  was  inspired  by  Darwin’s  theory  of  evolution. 
In  contrast  to  conventional  expert  systems  that  specialize  or  complexify  given  hypotheses  referring 
exclusively  to  new  cases  and  determined  rules,  GA  simulate  biological  evolution  by  repeatedly  mutating 
and  recombining  parts  of  the  existing  hypotheses.  This  is  motivated  by  the  hope  that  the  modified  ones 
will  be  better  than  the  old  ones.  After  each  mutation,  the  new  hypotheses  are  evaluated  using  some 
previously  fixed  fitness  criteria.  Thus  GA  are  able  to  identify  good  hypotheses  from  spaces  of  candidates. 
So  far  they  have  been  used  successfully,  for  instance,  to  improve  the  topology  of  artificial  neural 
networks,  to  learn  rules  for  robot  control,  and  for  a  number  of  optimization  tasks. 

5.2  Evaluation  of  Techniques 

As  soon  as  a  situation  in  AAW  becomes  really  delicate,  human  assessment  is  essential.  But  operators  are 
capable  of  doing  this  job  properly  only  if  they  can  retain  an  adequate  situational  awareness  despite  being 
supported.  Therefore,  there  has  to  be  a  sufficient  amount  of  transparency  with  the  applied  support  system. 
Evidence  for  this  can  also  be  found  in  the  outcomes  of  an  earlier  study  [Boiler  2000]  in  which  a  prototypal 
rule-based  support  system  was  evaluated  by  Navy  personnel.  The  test  persons  made  complaints  about 
being  out  of  the  loop.  Whenever  critical  situations  evolved  they  felt  like  being  reduced  to  confirming 
automated  application  of  rules  by  clicking,  without  having  the  chance  to  get  an  understanding  of  what  was 
happening  around  them  and  of  what  they,  respectively  the  support  system,  were  doing. 

Hence  the  applied  techniques  in  the  new  support  system  must  enable  the  operator  to  retain  an  adequate 
situational  awareness.  Certain  knowledge-based  techniques  can  provide  auxiliary  information  about  the 
way  that  led  to  their  conclusions,  others  cannot.  For  this  reason,  neural  networks  are  not  considered  a  good 
strategy  as  they  lack  any  transparency.  Generic  algorithms  are  also  not  appropriate  due  to  their  inherent 
mutation  that  makes  them  unpredictable. 
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6.0  THE  EMPIRICAL  APPROACH  TO  KNOWLEDGE  ACQUISITION 

In  order  to  find  out  which  events  and  attributes  are  relevant  to  object  identification,  intent  recognition, 
and  threat  evaluation  in  AAW  an  empirical  study  with  experienced  operators  will  be  undertaken.  The  goal 
is  to  acquire,  describe,  formalize,  and  represent  practical  expert  knowledge.  This  is  needed  to  frame  a 
case-base,  which  is  necessary  to  operate  the  aforementioned  knowledge  representation  techniques. 
Additionally,  the  gained  data  shall  be  used  for  general  task  and  information  analyses  to  yield  human 
centered  design  approaches  for  visualization  and  handling. 

To  act  as  experimentation  facility  and  demonstrator  a  modular  and  extendible  laboratory  test  bed  has  been 
developed.  It  consists  of  three  parts:  data  generation,  human-machine  interface  (HMI)  for  subject 
interaction,  and  experiment  control. 

The  data  generation  is  done  with  STAGE  (Scenario  Toolkit  And  Generation  Environment),  which  is  a 
commercial-off-the-shelf  software  used  by  numerous  suppliers  of  military  equipment.  With  STAGE 
simulated  scenarios  for  military  applications  can  be  created  graphically  and  run  in  real  time, 
whereby  quasi-realistic  sensor  data  are  provided.  The  scenarios  for  the  experiments  are  created  in  close 
cooperation  with  a  German  Navy  school.  The  experiments  can  only  be  expected  to  provide  useful  and 
enfolding  information  if  all  relevant  points  are  included  in  the  scenarios.  Therefore,  they  must  include 
situations  (Figure  9)  that  typically  appear  in  navies’  future  scope  of  duties,  e.g.: 

•  Operations  in  coastline  regions 

•  Presence  of  airways 

•  Mixed  environments  with  friendly,  hostile,  and  neutral  forces 

•  Additional  sensor  data  provided  by  AW  ACS 

•  Ambiguous  parameters  and  behaviors 

•  Incomplete  data 


Figure  9:  A  Small  Sample  Scenario  with  Airways. 
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The  HMI,  which  the  subjects  will  face,  is  designed  in  the  style  of  presently  existing  systems 
(Figure  10)  to  make  its  functionality  and  usability  easy  to  learn  for  people  who  work  with  current  systems. 
The  presentation  of  information  takes  place  in  two  display  areas:  plan  position  indicator  (PPI)  and  track 
information  display.  A  single  object  can  be  marked  by  clicking  on  its  symbol  within  the  PPI.  Then  all 
available  sensor  data  belonging  to  that  object  are  displayed  in  the  track  information  area  in  an 
alphanumerical  manner. 


Figure  10:  Graphical  Desktop  of  the  Laboratory  Test  Bed. 


Other  features  of  the  HMI  include: 

•  Buttons  to  set  ID  and  type 

•  Buttons  to  scroll  left,  right,  up,  down,  and  to  the  center 

•  Hook  function 

•  Zoomable  range  from  8  to  5 12  nautical  miles 

•  Popup  menus  to  set  ID  and  type 

•  Optional  map  background 

•  Track  numbers  shown  optionally 

•  Speed  /  direction  vectors  of  adaptable  length 

•  0,  5,  10,  20,  or  40  history  points  displayable  for  all  tracks 

•  Selection,  identification,  and  classification  of  multiple  tracks  at  once 

•  Voice  control 
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In  the  experiments  the  task  of  the  military  experts  will  be  to  make  decisions  while  facing  simulated 
scenarios.  All  events  and  parameter  constellations  as  well  as  the  whole  user  behavior  are  logged. 
That  brings  us  to  the  third  component  of  the  test  bed:  the  experiment  control.  Accompanying  each  decision 
the  operators  are  requested  to  explain  which  of  the  object’s  attributes  and  behaviors  contributed  to  their 
decision.  Conceivable  explanations  include  the  following: 

•  Origin  of  track 

•  Course/speed  changes 

•  IFF  response 

•  Electronic  emissions 

•  Rate  of  ascent  /  descent 

•  Suitability  to  airway  pattern 

•  Flying  in-  /  outbound 

•  Constellation  /  formation  of  multiple  tracks 

Human  decision  making  is  difficult  to  comprehend,  therefore  this  list  might  lack  some  relevant  points. 
To  prevent  the  possible  explanations  from  being  limited  to  what  the  authors  can  think  of  at  the  time  of 
experiment  design,  verbal  annotation  (“thinking  aloud”)  of  each  decision  is  asked  for,  rather  than  filling  in 
some  kind  of  multiple  choice  questionnaire.  In  addition  to  the  structured  logging,  which  is  to  provide  data 
that  can  be  analyzed  with  algorithmic  means,  the  complete  screen  activity  is  recorded  with  a  screen 
recorder.  In  the  experiments,  a  variant  of  this  hardware  that  has  been  specially  modified  is  deployed. 
This  variant  is  capable  of  doing  voice  recording  simultaneously  to  screen  capture.  This  results  in 
synchronous  videos  of  the  whole  experiment,  which  can  be  reviewed  by  investigator  and  subject 
retrospectively  to  find  out  and  supplement  what  has  not  been  explained  so  far. 


7.0  CONCLUSION  AND  OUTLOOK 

The  work  human  decision-makers  onboard  naval  platforms  have  to  perform  is  becoming  more  demanding 
due  to  new  types  of  missions  as  well  as  advanced  warplanes,  more  accurate  weapons,  and  long-range 
sensors.  Hence  providing  operators  with  adequate  support  is  an  essential  requirement  for  future  Navy 
systems.  Because  of  the  high  complexity  of  most  operator  tasks  full  automation  is  not  advisable. 
The  final  decisions  must  be  made  by  human  operators.  Appropriately  helping  them  to  create  a  correct 
understanding  of  the  situation  can  make  the  difference  between  a  catastrophe  and  doing  exactly  the  right 
thing. 

Some  particular  behavior  patterns  can  be  modeled  best  by  rule-based  AI,  for  others  more  sophisticated 
approaches  have  to  be  considered.  Five  techniques  have  been  described  within  this  paper,  whereof  three 
shall  be  deemed  to  be  potential  constituents  of  the  aspired  support  system.  They  all  have  in  common  that 
enfolding  case-bases  are  needed  to  become  able  to  apply  them  beneficially.  A  conceivable  way  to  obtain 
such  a  case-base  is  empirical  knowledge  acquisition  by  means  of  a  laboratory  test  bed.  Furthermore, 
means  to  structure  the  so  elicited  knowledge  have  to  be  chosen  and  evaluated.  Utilizing  the  methodology 
introduced  here  could  yield  helpful  support  for  AAW  operators,  but  yet  there  is  much  more  research  to  be 
done. 
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